Transferring Application Software from a Physical to a Virtual Computer System

ABSTRACT

A method and apparatus for testing application software for installation on hardware platforms. A software tool is verified using a tool qualifier module running on a physical computer system that has a processor of a first processor type and an operating system of a first operating type. A benchmark performance is established for the physical computer system. A virtual machine in a virtual computer system is verified using a qualifier module to determine whether its performance is consistent with the benchmark performance to form a qualified virtual machine. The virtual computer system has a configuration that comprises a processor of a second type; an operating system of a second operating type; the virtual machine emulating the processor of the first processor type; and an operating system of the first operating type. The software tool can be transferred from the physical computer system to the qualified virtual machine without requiring requalification.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to the following patent applications:entitled “Testing Application Programs Using a Virtual Machine”, Ser.No. ______, attorney docket no. 15-1436-US-NP1; and entitled“Transferring Application Software Between Virtual Machines”, Ser. No.______, attorney docket no. 15-1436-US-NP2; filed even date hereof andassigned to the same assignee. Each of the aforementioned applicationsis incorporated herein by reference in its entirety.

BACKGROUND INFORMATION 1. Field

The present disclosure relates generally to application programs and, inparticular, to testing application programs. Still more particularly,the present disclosure relates to a method and apparatus for testingapplication programs using application testing tools that are run onvirtual machines.

2. Background

Many different types of complex systems are operated using applicationprograms that require certification by one or more agencies. Forexample, without limitation, commercial aircraft, military aircraft,spacecraft, autonomous vehicles, nuclear power plants, medicalequipment, and other types of complex systems require certification ofdifferent application programs. However, as application programsincrease in size and complexity, it becomes increasing difficult tomanually test these application programs and then verify that theresults of this testing are accurate and complete.

Consequently, the testing and analysis of testing results areincreasingly being performed through automated processes involvingspecialized computer programs, which may be referred to as applicationtesting tools. But these application testing tools may themselves be asource of error that leads to testing results that are inaccurate,incomplete, or both.

Thus, these application testing tools may need to be verified andvalidated before they can be used to test application programs. Forexample, without limitation, before an application testing tool may beused on a particular hardware configuration to test an applicationprogram, the application testing tool may need to be qualified.Qualifying the application testing tool ensures that the applicationtesting tool behaves as specified under all operational conditions. Forexample, an application testing tool may be qualified by verifying thatthe application testing tool meets all expected requirements when theapplication testing tool is run using a particular hardware and softwareconfiguration.

However, given the rate that computer hardware and software is evolving,it may be difficult to establish and maintain specific hardwareconfigurations for running the application testing tools used to testapplication programs. In particular, requalifying an application testingtool after modifications, upgrades, or replacements have been made withrespect to the original hardware and software configuration used toqualify the application testing tool, may be more difficult,time-consuming, and expensive than desired. Therefore, it would bedesirable to have a method and apparatus that take into account at leastsome of the issues discussed above, as well as other possible issues.

SUMMARY

In one illustrative embodiment, an apparatus comprises a tool qualifiermodule, a physical computer system, a virtual computer system, abenchmark module, and a qualifier module. The physical computer systemcomprises a first processor of a first processor type and a firstoperating system of a first operating type running on the firstprocessor of the first processor type. The tool qualifier module is runon the physical computer system to verify an application testing toolusing tool qualification data to qualify the application testing tool.The benchmark module establishes a benchmark performance for thephysical computer system. The virtual computer system has aconfiguration that comprises a second processor of a second processortype; a second operating system of a second operating type running onthe second processor of the second processor type; a virtual machineemulating the first processor of the first processor type and running onthe second processor of the second processor type; and a third operatingsystem of the first operating type running on the virtual machine. Thequalifier module is used for verifying the virtual machine to determinewhether the performance of the virtual machine is consistent with thebenchmark performance of the physical computer system within selectedtolerances to thereby form a qualified virtual machine such thattransferring the application testing tool after the application testingtool has been qualified from the physical computer system to thequalified virtual machine on the virtual computer system does notrequire requalifying the application testing tool.

In another illustrative embodiment, a method is provided for testingapplication software for installation on a hardware platform usingvirtual computer systems. An application testing tool is verified usinga tool qualifier module running on a physical computer system having afirst processor of a first processor type and a first operating systemof a first operating type running on the first processor of the firstprocessor type. A benchmark performance is established for the physicalcomputer system. A virtual machine in a virtual computer system isverified using a qualifier module to determine whether a performance ofthe virtual machine is consistent with the benchmark performance of thephysical computer system within selected tolerances to thereby form aqualified virtual machine. The virtual computer system has aconfiguration that comprises a second processor of a second processortype; a second operating system of a second operating type running onthe second processor of the second processor type; the virtual machineemulating the first processor of the first processor type and running onthe second processor of the second processor type; and a third operatingsystem of the first operating type running on the virtual machine. Afterthe application testing tool has been qualified on the physical computersystem, the application testing tool is transferred from the physicalcomputer system to the qualified virtual machine on the virtual computersystem. Using the application testing tool on the second virtualcomputer system to test an application program that is to be installedon the hardware platform does not require requalifying the applicationtesting tool.

In yet another illustrative embodiment, a method is provided for testingapplication software for installation on an aircraft hardware platformusing virtual computer systems. An application testing tool is qualifiedusing a tool qualifier module running on a physical computer systemhaving a first processor of a first processor type and a first operatingsystem of a first operating type running on the first processor of thefirst processor type. A benchmark performance is established for thephysical computer system. The virtual computer system has aconfiguration that comprises a second processor of a second processortype; a second operating system of a second operating type running onthe second processor of the second processor type; the virtual machineemulating the first processor of the first processor type and running onthe second processor of the second processor type; and a third operatingsystem of the first operating type running on the virtual machine. Thevirtual machine in the virtual computer system is verified using aqualifier module to determine whether the performance of the virtualmachine is consistent the benchmark performance of the physical computersystem within selected tolerances to thereby form a qualified virtualmachine. After the application testing tool has been verified, theapplication testing tool is transferred from the physical computersystem to the qualified virtual machine on the virtual computer system.Using the application testing tool on the virtual computer system totest an application program that is to be installed on the aircrafthardware platform does not require requalifying the application testingtool. An application program is tested for installation on the aircrafthardware platform using the application testing tool running on thevirtual computer system to determine whether the application program iscertifiable without requalifying the application testing tool for use onthe virtual computer system. The application program is installed on theaircraft hardware platform in response to a determination that theapplication program is certifiable.

The features and functions can be achieved independently in variousembodiments of the present disclosure or may be combined in yet otherembodiments in which further details can be seen with reference to thefollowing description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the illustrativeembodiments are set forth in the appended claims. The illustrativeembodiments, however, as well as a preferred mode of use, furtherobjectives and features thereof, will best be understood by reference tothe following detailed description of an illustrative embodiment of thepresent disclosure when read in conjunction with the accompanyingdrawings, wherein:

FIG. 1 is an illustration of a virtual computer system in the form of ablock diagram in accordance with an illustrative embodiment;

FIG. 2 is an illustration of a transfer of an application testing toolfrom a first virtual computer system to a second virtual computer systemin the form of a block diagram in accordance with an illustrativeembodiment;

FIG. 3 is an illustration of a transfer of an application testing toolfrom a physical computer system to a virtual computer system in the formof a block diagram in accordance with an illustrative embodiment;

FIG. 4 is an illustration of a process for testing application softwarefor installation on a hardware platform using a virtual computer systemin the form of a flowchart in accordance with an illustrativeembodiment;

FIG. 5 is an illustration of a process for testing application softwarefor installation on a hardware platform using a virtual computer systemin the form of a flowchart in accordance with an illustrativeembodiment;

FIG. 6 is an illustration of a process for testing application softwarefor installation on a hardware platform using a virtual computer systemin the form of a flowchart in accordance with an illustrativeembodiment;

FIG. 7 is an illustration of a process for creating a tool qualificationvirtual machine in the form of a flowchart in accordance with anillustrative embodiment;

FIG. 8 is an illustration of a process for qualifying a plurality ofsoftware tools in the form of a flowchart in accordance with anillustrative embodiment;

FIG. 9 is an illustration of a process for verifying an applicationtesting tool in the form of a flowchart in accordance with anillustrative embodiment;

FIG. 10 is an illustration of a process for preparing applicationsoftware for installation on a hardware platform in the form of aflowchart in accordance with an illustrative embodiment;

FIG. 11 is an illustration of a data processing system in the form of ablock diagram in accordance with an illustrative embodiment;

FIG. 12 is an illustration of an aircraft manufacturing and servicemethod in the form of a block diagram in accordance with an illustrativeembodiment; and

FIG. 13 is an illustration of an aircraft in the form of a block diagramin accordance with an illustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiments take into account different considerations.In particular, the illustrative embodiments take into account that itmay be desirable to qualify an application testing tool that is used totest application programs and to tie that qualification to a particularcomputer hardware and software configuration used to perform thequalification. Tying this qualification to the computer hardware andsoftware configuration may avoid the risk of errors that may beintroduced due to a hardware change or a software change.

The illustrative embodiments take into account that many differentsoftware patches and software upgrades may be implemented over the lifeof a system that is in service for many years or even many decades. Forexample, without limitation, many different application programs may beinstalled on an aircraft over the life of the aircraft. In some cases,the application testing tools used to test these application programs atthe beginning of the life of an aircraft may be unable to run on the newhardware and software later in the life of the aircraft. In some case,the original hardware and software configuration used to qualify anapplication testing tool at the beginning of the life of the aircraftmay be unavailable later in the life of the aircraft. Further,recreating this original hardware and software configuration may beimpossible later in the life of the aircraft.

Thus, the illustrative embodiments provide a method and apparatus foreasily and quickly qualifying and requalifying application testing toolsusing virtual machines. In particular, the application testing tools arehosted on virtual machines that are configured to emulate baselinehardware and software configurations in a manner that is unaffected byinevitable changes in the hardware and software of physical computersystems that occur over time. In this manner, the illustrativeembodiments provide a virtual machine configured to allow for softwaretools, such as an application testing tool, to be executed in thevirtual machine in a manner that produces guaranteed, consistent resultsacross a variety of physical computer system hosts. The illustrativeembodiments provide a virtual machine for tool qualification thateliminates the need to maintain a particular hardware configuration fora long period of time, such as decades, or to requalify an applicationtesting tool each time the application testing tool is used on aphysical computer system having a new hardware configuration.

Further, the illustrative embodiments provide a method and apparatusthat reduces the overall time, effort, and cost associated withrequalifying application testing tools, which have already beenqualified on a virtual machine running on a first hardwareconfiguration, on that same virtual machine running on a second hardwareconfiguration. In particular, only a verification of the equivalencebetween the virtual machine running on the first hardware configurationand the virtual machine running on the second hardware configurationneeds to be performed. Once verification of the virtual machine runningon the second hardware configuration has been satisfactorily performed,this verification will automatically apply to all software that is runon the virtual machine. In other words, the application testing toolsthat were previously qualified for the virtual machine running on thefirst hardware configuration will not need to be individuallyrequalified for the virtual machine running on the second hardwareconfiguration.

Referring now to the figures, the same reference numeral may be used inmore than one figure. This reuse of a reference numeral in differentfigures represents the same element in the different figures.

With reference to FIG. 1, an illustration of a virtual computer systemis depicted in the form of a block diagram in accordance with anillustrative embodiment. In this illustrative example, virtual computersystem 100 comprises physical computer system 102 that includes firstprocessor 104 of first processor type 106. First operating system 108 offirst operating type 110 may be run on first processor 104 of firstprocessor type 106. In some cases, an operating type, such as firstoperating type 110, may be referred to as an operating system type.

Virtual computer system 100 also comprises virtual machine 112 that isrun on first processor 104. Virtual machine 112 emulates secondprocessor 114 of second processor type 116. In particular, virtualmachine 112 emulates hardware and software in a manner that allowsemulation of second processor 114 of second processor type 116.

In this manner, virtual machine 112 contains an emulation of secondprocessor 114 of second processor type 116. In one illustrative example,second processor 114 may be referred to as a baseline processor andsecond processor type 116 may be referred to as a baseline processortype.

Second operating system 118 of second operating type 120 may be run onvirtual machine 112. In particular, virtual machine 112 is a computerprogram that allows simulation of the execution of second operatingsystem 118 of second operating type 120 and, in some cases, one or moreother programs, by second processor 114 emulated in the virtual machinerunning on first processor 104 of physical computer system 102.

In these illustrative examples, virtual machine 112 is configured to runin virtual time. In particular, virtual machine 112 may be run invirtual time, independently of any physical clock or processor-basedclock in physical computer system 102.

In one illustrative example, first processor type 106 is different fromsecond processor type 116. Further, in this illustrative example, firstoperating type 110 is different from second operating type 120. Forexample, without limitation, first processor type 106 and secondprocessor type 116 may belong to different processor families, may bedifferent releases, may comprise different processor architectures, maybe different models, may be different generations of a processor, may bebuilt by different manufacturers, may be different in some other way, ora combination thereof.

Similarly, first operating type 110 and second operating type 120 maybelong to different operating system families, may be differentreleases, may comprise different operating system architectures, may bedifferent models, may be different generations of an operating system,may be developed by different software developers, may be different insome other way, or a combination thereof.

As another example, without limitation, second processor type 116 may benewer than or belong to a newer generation of processor types than firstprocessor type 106. Similarly, second operating type 120 may be newerthan or belong to a newer generation of operating system types thanfirst operating type 110.

Together, the physical layer defined by first processor type 106 andfirst operating type 110 and the virtual layer defined by secondprocessor type 116 and second operating type 120 define configuration121 of virtual computer system 100. Thus, configuration 121 is ahardware and software configuration composed of both a physical layerand a virtual layer.

Virtual machine 112 is run as an emulation of a baseline hardware andsoftware configuration entirely through software. This emulation isconfigured to be unaffected by future changes in the hardware andsoftware of physical computer system 102 over time. For example, withoutlimitation, virtual machine 112 may be adapted and modified over time tobe compatible with different physical computer systems having differenthardware and software configurations without the emulation of thebaseline hardware and software configuration by virtual machine 112being unaffected.

Tool qualifier module 122 may be run on virtual machine 112 using secondoperating system 118 to perform verification of software tool 123. Insome cases, tool qualifier module 122 may also be used to performvalidation of software tool 123. Software tool 123 may be any softwaretool that is configured for use in testing, creating, developing, orotherwise using application program 126. Software tool 123 may take anumber of different forms such as, but not limited to, an applicationtesting tool, a compiler tool, a developer tool, or some other type oftool. In one illustrative example, software tool 123 takes the form ofapplication testing tool 124. Performing this verification includesconfirming that application testing tool 124 satisfies all expectedrequirements for application testing tool 124. Application testing tool124 may be used to, for example, without limitation, create and developthe program code for application program 126.

In particular, tool qualifier module 122 may verify application testingtool 124 to qualify application testing tool 124 for use in testingapplication program 126. In particular, tool qualifier module 122 isused to verify application testing tool 124 before application testingtool 124 is run on virtual machine 112 using second operating system 118to test application program 126. Application program 126 may be a newcomputer program, a software patch, a software upgrade, or some othertype of computer program that is to be installed on a hardware platform.

In these illustrative examples, tool qualifier module 122 verifiesapplication testing tool 124 using tool qualification data 128 toqualify that application testing tool 124 runs on virtual machine 112 inaccordance with all expected requirements. Tool qualification data 128may include at least one of tool source data, regression test inputdata, regression test output data, or some other type of data.

As used herein, the phrase “at least one of,” when used with a list ofitems, means different combinations of one or more of the listed itemsmay be used and only one of the items in the list may be needed. Theitem may be a particular object, thing, step, operation, process, orcategory. In other words, “at least one of” means any combination ofitems or number of items may be used from the list, but not all of theitems in the list may be required.

For example, without limitation, “at least one of item A, item B, oritem C” or “at least one of item A, item B, and item C” may mean item A;item A and item B; item B; item A, item B, and item C; item B and itemC; or item A and C. In some cases, “at least one of item A, item B, oritem C” or “at least one of item A, item B, and item C” may mean, but isnot limited to, two of item A, one of item B, and ten of item C; four ofitem B and seven of item C; or some other suitable combination.

Application testing tool 124 may be run on virtual machine 112 usingsecond operating system 118 to determine whether application program 126is certifiable. In one illustrative example, application testing tool124 tests whether application program 126 provides expected output datain response to a particular set of inputs to application program 126, todetermine whether application program 126 is certifiable. In some cases,application testing tool 124 uses application testing data 129 to testapplication program 126. Application testing data 129 may include atleast one of application source code, regression test input data,regression test output data, or some other type of data.

Hosting application testing tool 124 on virtual machine 112 allowsapplication testing tool 124 to be run on physical computer system 102independently of first processor 104 and first operating system 108.Running virtual machine 112, which emulates second processor 114 ofsecond processor type 116 on first processor 104 of first processor type106, enables execution of application testing tool 124 through secondoperating system 118 of second operating type 120 running on virtualmachine 112 even when application testing tool 124 is incapable offunctioning correctly using first operating system 108 of firstoperating type 110 and when second operating type 120 of secondoperating system 118 is incompatible with first processor type 106 offirst processor 104.

Application program 126 is designed for use on a hardware platform thatis either connected to or embedded within a complex system. This complexsystem may take the form of, for example, without limitation, a vehicle,an autonomous vehicle, a nuclear power plant, a medical device, or someother type of system. In one illustrative example, the complex system isvehicle 130 and application program 126 is designed for use on vehiclehardware platform 132 onboard vehicle 130.

In some illustrative examples, virtual computer system 100 includesinterface connection 134 to vehicle hardware platform 132. Vehiclehardware platform 132 may take the form of one of an aircraft hardwareplatform, a spacecraft hardware platform, a watercraft hardwareplatform, a ground vehicle hardware platform, or some other type ofvehicle hardware platform.

Interface connection 134 allows for easy transfer of application program126 from virtual computer system 100 onto vehicle hardware platform 132for installation. As one illustrative example, in response to thesatisfactory testing of application program 126 by application testingtool 124, second operating system 118 may output application program 126over interface connection 134 to vehicle hardware platform 132 forinstallation. For example, if application testing tool 124 determinesthat application program 126 is certifiable, application testing tool124 may generate a certification result indicating that applicationprogram 126 is approved for installation on the hardware platform.

Depending on the implementation, interface connection 134 is implementedas either part of physical computer system 102 or connected to physicalcomputer system 102. Further, depending on the implementation, interfaceconnection 134 may be used to establish a wireless, wired, optical, orother type of connection between virtual computer system 100 and vehiclehardware platform 132. Interface connection 134 is made available tosecond operating system 118 through virtual machine 112. In otherillustrative examples, application program 126 may be installed onvehicle hardware platform 132 in some other manner.

In some illustrative examples, qualifier module 136 is run on firstprocessor 104 outside of virtual machine 112. Qualifier module 136 maybe used to perform verification of virtual machine 112. In particular,qualifier module 136 may use machine qualification data 138 to verifythat virtual machine 112 functions according to expected requirementswhile virtual machine 112 runs second operating system 118 on firstprocessor 104. Machine qualification data 138 may include at least oneof machine source data, regression test input data, regression testoutput data, or some other type of data.

Using virtual machine 112 to perform the verification and, in somecases, validation, of application testing tool 124 eliminates the needto maintain a physical computer system having a baseline hardware andsoftware configuration. In this manner, virtual machine 112 thatemulates second processor 114 of second processor type 116 may persistfor tool qualification even after the physical hardware platform orphysical hardware components used to implement processors of secondprocessor type 116 becomes obsolete. Virtual machine 112 may persist,for example, without limitation, for decades, but a particular hardwareplatform may become obsolete in only a few years.

In one illustrative example, virtual machine 112 is stored in the formof one or more software files in some type of data storage. For example,without limitation, virtual machine 112 may be stored as one or moresoftware files in cloud storage. Virtual machine 112 may be downloadableonto different physical host computer systems for use in qualifyingapplication testing tools. Virtual machine 112 may not require anymaintenance over the years even though the hardware configurations ofthe physical host computer systems may change over time.

In some illustrative examples, virtual machine 112 may be referred to asa tool qualification virtual machine. Running tool qualifier module 122on virtual machine 112, which is running on first processor 104 ofphysical computer system 102, transforms physical computer system 102into virtual computer system 100 that is a special-purpose computersystem for performing tool qualification and application testing.

With reference now to FIG. 2, an illustration of a transfer of anapplication testing tool from a first virtual computer system to asecond virtual computer system is depicted in the form of a blockdiagram in accordance with an illustrative embodiment. In thisillustrative example, application testing tool 124 from FIG. 1 may betransferred from a first virtual computer system having a firstconfiguration to a different virtual computer system having a differentconfiguration.

For example, the first virtual computer system having the firstconfiguration may be virtual computer system 100 from FIG. 1 havingconfiguration 121. In this illustrative example, virtual computer system100 includes physical computer system 102 and virtual machine 112, whichmay also be referred to as a first physical computer system and a firstvirtual machine, respectively.

Application testing tool 124 may be verified using tool qualifier module122 running on virtual computer system 100. Application testing tool 124may then be transferred from virtual computer system 100 to secondvirtual computer system 200 having second configuration 202.

Second virtual computer system 200 includes second physical computersystem 204. Second physical computer system 204 includes third processor206 of third processor type 208. Third operating system 210 of thirdoperating type 212 runs on third processor 206.

Second virtual computer system 200 also includes second virtual machine214. Similar to virtual machine 112, second virtual machine 214 emulatessecond processor 114 of second processor type 116. In this illustrativeexample, fourth operating system 216 of second operating type 120 runson second virtual machine 214. Fourth operating system 216 is the sametype of operating system as second operating system 118. However, inother illustrative examples, fourth operating system 216 may be someother type of operating system with which application testing tool 124is compatible.

Second configuration 202 of second virtual computer system 200 isdefined by both a physical layer and a virtual layer. The physical layeris defined by third processor type 208 and third operating type 212. Thevirtual layer is defined by second processor type 116 and secondoperating type 120.

Prior to the transfer of application testing tool 124 from virtualcomputer system 100 to second virtual computer system 200, benchmarkmodule 220 is used to establish benchmark performance 222 for virtualmachine 112. Benchmark module 220 may be run on first processor 104using first operating system 108 outside of virtual machine 112, whilevirtual machine 112 is run on first processor 104, to establishbenchmark performance 222 for virtual machine 112.

Qualifier module 136 may be used to verify that second virtual machine214 running on third processor 206 of second virtual computer system 200meets benchmark performance 222 of virtual machine 112 within selectedtolerances to thereby qualify second virtual machine 214. In otherwords, qualifier module 136 verifies that the performance of secondvirtual machine 214 is consistent with benchmark performance 222 ofvirtual machine 112. In other words, qualifier module 136 verifiesequivalence between virtual machine 112 and second virtual machine 214.Depending on the implementation, machine qualification data 138 may alsobe used to perform this verification. In this manner, a qualified secondvirtual machine is formed.

Once application testing tool 124 has been qualified on virtual machine112, transferring application testing tool 124 from virtual machine 112to second virtual machine 214 does not require requalifying applicationtesting tool 124 if second virtual machine 214 is already determined tobe a qualified second virtual machine. Savings in time and effort areachieved by eliminating the need to repeat verification of applicationtesting tool 124 on second virtual computer system 200.

With reference now to FIG. 3, an illustration of a transfer of anapplication testing tool from a physical computer system to a virtualcomputer system is depicted in the form of a block diagram in accordancewith an illustrative embodiment. In this illustrative example,application testing tool 124 from FIG. 1 may be transferred fromphysical computer system 102 from FIG. 1 to virtual computer system 300.

As depicted, physical computer system 102 includes first processor 104of first processor type 106. First operating system 108 of firstoperating type 110 is run on first processor 104.

In this illustrative example, application testing tool 124 may beverified using tool qualifier module 122. For example, tool qualifiermodule 122 may use tool qualification data 128 to perform verificationof application testing tool 124.

As depicted, virtual computer system 300 may include second physicalcomputer system 204 from FIG. 2 having third processor 206 of thirdprocessor type 208. In this illustrative example, third operating system210 of third operating type 212 is run on third processor 206 of thirdprocessor type 208. Further, virtual computer system 300 includesvirtual machine 302 that is run on third processor 206. Virtual Machine302 emulates first processor 104 of first processor type 106. Operatingsystem 304 of first operating type 110 is run on virtual machine 302.

Virtual computer system 300 has third configuration 306. Thirdconfiguration 306 of virtual computer system 300 is defined by both aphysical layer and a virtual layer. The physical layer is defined bythird processor type 208 and third operating type 212. The virtual layeris defined by first processor type 106 and first operating type 110.

Before application testing tool 124 is transferred from physicalcomputer system 102 to virtual computer system 300, benchmark module 220is used to establish benchmark performance 222 for physical computersystem 102. Qualifier module 136 then verifies virtual machine 302 usingbenchmark performance 222 to qualify the equivalence between physicalcomputer system 102 and virtual machine 302. In this manner, qualifiermodule 136 verifies that the performance of virtual machine 302 isconsistent with benchmark performance 222 for physical computer system102. In some cases, qualifier module 136 may also use machinequalification data 138 to perform the verification of virtual machine302.

Once virtual machine 302 has been qualified, or, in other words,determined to be a qualified virtual machine, application testing tool124 may be transferred from physical computer system 102 to virtualmachine 302. In particular, application testing tool 124 may then be runon virtual machine 302 without having to repeat the verification ofapplication testing tool 124. Savings in time and effort are achieved byeliminating the need to repeat verification of application testing toolon virtual computer system 300.

The illustrations in FIGS. 1-3 are not meant to imply physical orarchitectural limitations to the manner in which an illustrativeembodiment may be implemented. Other components in addition to or inplace of the ones illustrated may be used. Some components may beoptional. Also, the blocks are presented to illustrate some functionalcomponents. One or more of these blocks may be combined, divided, orcombined and divided into different blocks when implemented in anillustrative embodiment.

In some illustrative examples, the various modules described in FIGS.1-3 may be implemented using software, hardware, firmware, or acombination thereof. For example, without limitation, each of toolqualifier module 122, qualifier module 136, and benchmark module 220 maybe implemented using software, hardware, firmware, or a combinationthereof. When software is used, the operations performed by a module maybe implemented using, for example, without limitation, program codeconfigured to run on a processor unit. When firmware is used, theoperations performed by the module may be implemented using, forexample, without limitation, program code and data and stored inpersistent memory to run on a processor unit.

When hardware is employed, the hardware may include one or more circuitsthat operate to perform the operations performed by the module.Depending on the implementation, the hardware may take the form of acircuit system, an integrated circuit, an application specificintegrated circuit (ASIC), a programmable logic device, or some othersuitable type of hardware device configured to perform any number ofoperations.

A programmable logic device may be configured to perform certainoperations. The device may be permanently configured to perform theseoperations or may be reconfigurable. A programmable logic device maytake the form of, for example, without limitation, a programmable logicarray, a programmable array logic, a field programmable logic array, afield programmable gate array, or some other type of programmablehardware device.

A computer program may comprise at least one of a compiler, aninterpreter, a main program, a sub-program, a script, or some othersoftware part. In this manner, a virtual machine, such as virtualmachine 112, an application testing tool, such as application testingtool 124, and an application program, such as application program 126,may each comprise at least one of a compiler, an interpreter, a mainprogram, a sub-program, a script, or some other software part.

Further, depending on the implementation, a hardware configuration or ahardware software configuration may include at least a processor and anoperating system. In some cases, a hardware configuration or a hardwaresoftware configuration may include additional components such as, forexample, without limitation, at least one of memory, an input/outputdevice, or some other type of component.

In alternative illustrative examples, second physical computer system204 in FIG. 3 may have a different configuration that includes secondprocessor 114 of second processor type 116 from FIG. 1 with secondoperating system 118 of second operating type 120 from FIG. 1 running onsecond processor 114. Thus, in these alternative examples, virtualmachine 302 may be run on second processor 114 using second operatingsystem 118. Further, in these alternative examples, third operatingsystem 210 of third operating type 212 may be configured to run onvirtual machine 302.

With reference now to FIG. 4, an illustration of a process for testingapplication software for installation on a hardware platform using avirtual computer system is depicted in the form of a flowchart inaccordance with an illustrative embodiment. The process illustrated inFIG. 4 may be implemented for virtual computer system 100 in FIG. 1.

The process may begin by running a first operating system of a firstoperating type on a first processor of a first processor type in aphysical computer system (operation 400). In operation 400, the physicalcomputer system may be, for example, physical computer system 102 inFIG. 1 having first processor 104 of first processor type 106 in FIG. 1.

Next, a virtual machine that emulates a second processor of a secondprocessor type is run on the first processor of the first processor typein the physical computer system (operation 402). In operation 402, thevirtual machine may be virtual machine 112 in FIG. 1. A second operatingsystem of a second operating type is run on the virtual machine with thevirtual machine running on the first processor of the first processortype in the physical computer system (operation 404).

In one illustrative example, the virtual machine described in operations402 and 404 may already be qualified on the first processor. In otherwords, verification that the virtual machine operates in a manner thatmeets all expected requirements may have already been performed. Morespecifically, verification that the virtual machine emulates the secondprocessor of the second processor type according to requirements whenrun on the first processor using the first operating system may havealready been performed.

Thereafter, a verification of an application testing tool is performedon the virtual machine using tool qualification data to qualify theapplication testing tool before the application testing tool is run onthe virtual machine using the second operating system to test anapplication program (operation 406). Verification of the applicationtesting tool through the virtual machine is performed independently ofthe first processor of the first processor type. In other words, theapplication testing tool is verified with respect to the virtual machineindependently of the processor on which the virtual machine is run.

The application testing tool is then run on the virtual machine to testthe application program (operation 408), with the process terminatingthereafter. In operation 408, the application testing tool may be usedto test the application program prior to the application program beinginstalled on a hardware platform, such as a vehicle hardware platform.

In one illustrative example, the application testing tool is incapableof functioning using the first operating system of the first operatingtype. Additionally, the second operating type of the second operatingsystem is incompatible with the first processor type of the firstprocessor. Running the virtual machine that emulates the secondprocessor of the second processor type on the first processor of thefirst processor type enables execution of the application testing toolthrough the second operating system of the second operating type runningon the virtual machine despite these incompatibilities.

With reference now to FIG. 5, an illustration of a process for testingapplication software for installation on a hardware platform using avirtual computer system is depicted in the form of a flowchart inaccordance with an illustrative embodiment. The process illustrated inFIG. 5 may be implemented using virtual computer system 100 in FIG. 2and second virtual computer system 200 in FIG. 2.

The process may begin by verifying an application testing tool using atool qualifier module running on a first virtual computer system havinga first configuration that comprises a first processor of a firstprocessor type; a first operating system of a first operating typerunning on the first processor of the first processor type; a firstvirtual machine emulating a second processor of a second processor typeand running on the first processor of the first processor type; and asecond operating system of a second operating type running on the firstvirtual machine (operation 500). In operation 500, this first virtualcomputer system may be virtual computer system 100 described in FIG. 2.

Next, a benchmark performance is established for the first virtualmachine (operation 502). Operation 502 may be performed using, forexample, benchmark module 220 in FIG. 2.

Thereafter, a second virtual machine in a second virtual computer systemis verified using a qualifier module to determine whether a performanceof the second virtual machine is consistent with the benchmarkperformance of the first virtual machine within selected tolerances tothereby form a qualified second virtual machine, wherein the secondvirtual computer system has a second configuration that comprises athird processor of a third processor type; a third operating system of athird operating type running on the third processor of the thirdprocessor type; the second virtual machine emulating the secondprocessor of the second processor type and running on the thirdprocessor of the third processor type; and a fourth operating system ofthe second operating type running on the second virtual machine(operation 504). The verification performed in operation 504 may beperformed using qualifier module 136 in FIG. 2.

In operation 504, the second virtual computer system may be, forexample, without limitation, second virtual computer system 200 in FIG.2. In some cases, the selected tolerances in operation 504 may be suchthat verification of the second virtual machine requires that the secondvirtual machine running on the third processor behave exactly as thefirst virtual machine that runs on the first processor.

After the application testing tool has been verified, the applicationtesting tool is transferred from the first virtual machine on the firstvirtual computer system to the qualified second virtual machine on thesecond virtual computer system, wherein using the application testingtool on the second virtual computer system to test an applicationprogram that is to be installed on a hardware platform does not requirerequalifying the application testing tool (operation 506), with theprocess terminating thereafter.

With reference now to FIG. 6, an illustration of a process for testingapplication software for installation on a hardware platform using avirtual computer system is depicted in the form of a flowchart inaccordance with an illustrative embodiment. The process illustrated inFIG. 6 may be implemented using physical computer system 102 in FIG. 1and virtual computer system 300 in FIG. 3.

The process begins by verifying an application testing tool using a toolqualifier module running on a physical computer system having a firstprocessor of a first processor type and a first operating system of afirst operating type running on the first processor of the firstprocessor type (operation 600). In operation 600, the tool qualifiermodule may be tool qualifier module 122 in FIG. 3.

Next, a benchmark performance is established for the physical processorsystem using a benchmark module (operation 602). In operation 602, thebenchmark module may be, for example, benchmark module 220 in FIG. 3.

Thereafter, a virtual machine in a virtual computer system is verifiedusing a qualifier module to determine whether the performance of thevirtual machine is consistent with the benchmark performance withinselected tolerances to thereby form a qualified virtual machine, whereinthe virtual computer system has a configuration that comprises a secondprocessor of a second processor type; a second operating system of asecond operating type running on the second processor of the secondprocessor type; the virtual machine emulating the first processor of thefirst processor type and running on the second processor of the secondprocessor type; and a third operating system of the first operating typerunning on the virtual machine (operation 604). In operation 604, thequalifier module may be qualifier module 136 in FIG. 3.

Next, after the application testing tool has been verified, theapplication testing tool is transferred from the physical computersystem to the qualified virtual machine on the virtual computer system,wherein using the application testing tool on the virtual computersystem to test an application program that is to be installed on thevehicle hardware platform does not require requalifying the applicationtesting tool (operation 606), with the process terminating thereafter.

With reference now to FIG. 7, an illustration of a process for creatinga tool qualification virtual machine is depicted in the form of aflowchart in accordance with an illustrative embodiment. The processillustrated in FIG. 7 may be used to create, for example, virtualmachine 112 described in FIGS. 1-2, virtual machine 302 in FIG. 3 orboth.

The process may begin by selecting a baseline processor of a baselineprocessor type (operation 700). In one illustrative example, inoperation 700, the baseline processor type may be the same processortype used in a vehicle hardware platform that is used onboard a vehicleor some other type of complex system. In other cases, the baselineprocessor type may be selected based on some other type of baselinehardware configuration or baseline hardware and software configuration.

Thereafter, a host physical computer system is selected (operation 702).Next, a virtual machine is created to emulate the baseline processor ofthe baseline processor type when the virtual machine is run on the hostphysical computer system (operation 704). In particular, the virtualmachine may be created to be compatible with the hardware and softwareconfiguration of the host physical computer but to emulate the baselineprocessor of the baseline processor type.

Further, in operation 704, the virtual machine is created such that thevirtual machine can host a desired set of software tools that are to beused for certifying software. For example, the virtual machine iscreated such that the virtual machine can host a variety of applicationtesting tools for testing application programs to determine whetherthese application programs are certifiable.

Thereafter, the virtual machine is run on the host physical computersystem (operation 706). A determination is made as to whether thevirtual machine behaves in accordance with all requirements (operation708). If the virtual machine does not behave in accordance with allrequirements, the virtual machine is modified (operation 710), with theprocess then returning to operation 706 described above. Operation 706is performed by running the modified virtual machine on the selectedhost physical computer system.

With reference again to operation 708, if the virtual machine doesbehave in accordance with all requirements, a final version of thevirtual machine is stored in cloud storage for future tool qualification(operation 712), with the process terminating thereafter. This finalversion of the virtual machine may be referred to as a toolqualification virtual machine.

With reference now to FIG. 8, an illustration of a process forqualifying a plurality of software tools is depicted in accordance withan illustrative embodiment. The process illustrated in FIG. 8 may beperformed using a virtual machine, such as a virtual machine 112described in FIGS. 1-2, virtual machine 302, or both.

The process begins by downloading a virtual machine that is configuredto host a plurality of application testing tools from cloud storage ontoa physical computer system in which the application testing tools havealready been qualified with respect to the virtual machine (operation800). Next, a verification of the virtual machine running on thephysical computer system is performed based on a benchmark performancefor the virtual machine (operation 802). The plurality of applicationtesting tools are then run on the virtual machine without requiring thateach of the plurality of application testing be requalified (operation804), with the process terminating thereafter.

With reference now to FIG. 9, an illustration of a process for verifyingan application testing tool is depicted in the form of a flowchart inaccordance with an illustrative embodiment. The process illustrated inFIG. 9 may be used to implement operation 406 in FIG. 4. In particular,this process may be used to verify, for example, application testingtool 124 in FIG. 1. Further, this process may be implemented usingvirtual computer system 100 in FIG. 1.

The process begins by preparing tool qualification data for use inverifying the application testing tool (operation 900). In operation900, the tool qualification data may include at least one of tool sourcedata, regression test input data, regression test output data, or someother type of input data.

Next, a determination is made as to whether the application testing toolpasses a verification test using tool qualification data (operation902). If the application testing tool passes the verification test, apassing result is generated (operation 904), with the processterminating thereafter.

In some cases, operation 900 may also include validating the applicationtesting tool. Thus, in these cases, operation 902 may includedetermining whether the application testing tool passes both theverification test and a validation test.

Otherwise, the application testing tool is modified to form a modifiedapplication testing tool (operation 906). Thereafter, new toolqualification data is prepared for use in verifying the modifiedapplication testing tool (operation 908). In operation 908, preparingthe new tool qualification data may include, for example, replacing thetool source data with new tool source data based on the modifiedapplication testing tool. In other illustrative examples, preparing thenew tool qualification data may also include generating at least one ofnew regression test input data, new regression test output data, or someother type of new data for testing. The process then returns tooperation 902 described above to determine whether the modifiedapplication testing tool passes the verification test using the new toolqualification data.

With reference now to FIG. 10, an illustration of a process forpreparing application software for installation on a hardware platformin the form of a flowchart is depicted in accordance with anillustrative embodiment. The process illustrated in FIG. 10 may beimplemented to test and install application program 126 in a hardwareplatform, such as vehicle hardware platform 132 in FIG. 1.

The process may begin by preparing application testing data for use intesting an application program (operation 1000). In operation 1000, theapplication testing data may include at least one of application sourcedata, regression test input data, regression test output data, or someother type of input data.

An application testing tool is run on a virtual machine in which thevirtual machine is run using a first operating system of a firstoperating type on a first processor of a first processor type in aphysical computer system, using a second operating system of a secondoperating type to test whether the application program is certifiable(operation 1002). In this manner, in operation 1002, the applicationtesting tool tests the application program. A determination is made asto whether the application program passes a certification test using theapplication testing data (operation 1004).

If the application program does not pass the certification test, theapplication program is modified to form a modified application program(operation 1006). Thereafter, new application testing data is preparedfor use in testing the modified application program (operation 1008).The process then returns to operation 1004 described above to determinewhether the modified application program passes the certification test.

In operation 1008, preparing the new application testing data mayinclude, for example, replacing the application source data prepared inoperation 1000 with new application source data based on the modifiedapplication program. In other illustrative examples, preparing the newapplication testing data may also include generating at least one of newregression test input data, new regression test output data, or someother type of new data for testing.

With reference again to operation 1004, if the application programpasses the certification test, a passing result is generated (operation1010). Next, the physical computer system is connected to a hardwareplatform via an interface connection (operation 1012). The virtualmachine provides connectivity to the physical interface connection forthe second operating system running on the second processor emulated inthe virtual machine. The application program is then output over theinterface connection for installation on the hardware platform(operation 1014), with the process terminating thereafter.

The flowcharts and block diagrams in the different depicted embodimentsillustrate the architecture, functionality, and operation of somepossible implementations of apparatuses and methods in an illustrativeembodiment. In this regard, each block in the flowcharts or blockdiagrams may represent a module, a segment, a function, and/or a portionof an operation or step.

In some alternative implementations of an illustrative embodiment, thefunction or functions noted in the blocks may occur out of the ordernoted in the figures. For example, in some cases, two blocks shown insuccession may be executed substantially concurrently, or the blocks maysometimes be performed in the reverse order, depending upon thefunctionality involved. Also, other blocks may be added in addition tothe illustrated blocks in a flowchart or block diagram.

For example, in other illustrative examples, the process described inFIG. 5 may include creating the first virtual machine at some pointprior to operation 500 and creating the second virtual machine at somepoint prior to operation 504. For example, the first virtual machine maybe created such that the virtual machine emulates the second processorof the second processor type when the first virtual machine is run onany processor of the first processor type. Further, the second virtualmachine may be created such that the second virtual machine emulates thesecond processor of the second processor type when the second virtualmachine is run on any processor of the third processor type.

Turning now to FIG. 11, an illustration of a data processing system inthe form of a block diagram is depicted in accordance with anillustrative embodiment. Data processing system 1100 may be used toimplement physical computer system 102 in FIG. 1, second physicalcomputer system 204 in FIG. 2, or both. As depicted, data processingsystem 1100 includes communications framework 1102, which providescommunications between processor unit 1104, storage devices 1106,communications unit 1108, input/output unit 1110, and display 1112. Insome cases, communications framework 1102 may be implemented as a bussystem.

Processor unit 1104 is configured to execute instructions for softwareto perform a number of operations. Processor unit 1104 may comprise anumber of processors, a multi-processor core, and/or some other type ofprocessor, depending on the implementation. In some cases, processorunit 1104 may take the form of a hardware unit, such as a circuitsystem, an application specific integrated circuit (ASIC), aprogrammable logic device, or some other suitable type of hardware unit.

Instructions for the operating system, applications, and/or programs runby processor unit 1104 may be located in storage devices 1106. Storagedevices 1106 may be in communication with processor unit 1104 throughcommunications framework 1102. As used herein, a storage device, alsoreferred to as a computer readable storage device, is any piece ofhardware capable of storing information on a temporary and/or permanentbasis. This information may include, but is not limited to, data,program code, and/or other information.

Memory 1114 and persistent storage 1116 are examples of storage devices1106. Memory 1114 may take the form of, for example, a random accessmemory or some type of volatile or non-volatile storage device.Persistent storage 1116 may comprise any number of components ordevices. For example, persistent storage 1116 may comprise a hard drive,a flash memory, a rewritable optical disk, a rewritable magnetic tape,or some combination of the above. The media used by persistent storage1116 may or may not be removable.

Communications unit 1108 allows data processing system 1100 tocommunicate with other data processing systems and/or devices.Communications unit 1108 may provide communications using physicaland/or wireless communications links.

Input/output unit 1110 allows input to be received from and output to besent to other devices connected to data processing system 1100. Forexample, input/output unit 1110 may allow user input to be receivedthrough a keyboard, a mouse, and/or some other type of input device. Asanother example, input/output unit 1110 may allow output to be sent to aprinter connected to data processing system 1100.

Display 1112 is configured to display information to a user. Display1112 may comprise, for example, without limitation, a monitor, a touchscreen, a laser display, a holographic display, a virtual displaydevice, and/or some other type of display device.

In this illustrative example, the processes of the differentillustrative embodiments may be performed by processor unit 1104 usingcomputer-implemented instructions. These instructions may be referred toas program code, computer usable program code, or computer readableprogram code and may be read and executed by one or more processors inprocessor unit 1104.

In these examples, program code 1118 is located in a functional form oncomputer readable media 1120, which is selectively removable, and may beloaded onto or transferred to data processing system 1100 for executionby processor unit 1104. Program code 1118 and computer readable media1120 together form computer program product 1122. In this illustrativeexample, computer readable media 1120 may be computer readable storagemedia 1124 or computer readable signal media 1126.

Computer readable storage media 1124 is a physical or tangible storagedevice used to store program code 1118 rather than a medium thatpropagates or transmits program code 1118. Computer readable storagemedia 1124 may be, for example, without limitation, an optical ormagnetic disk or a persistent storage device that is connected to dataprocessing system 1100.

Alternatively, program code 1118 may be transferred to data processingsystem 1100 using computer readable signal media 1126. Computer readablesignal media 1126 may be, for example, a propagated data signalcontaining program code 1118. This data signal may be an electromagneticsignal, an optical signal, and/or some other type of signal that can betransmitted over physical and/or wireless communications links.

The illustration of data processing system 1100 in FIG. 11 is not meantto provide architectural limitations to the manner in which theillustrative embodiments may be implemented. The different illustrativeembodiments may be implemented in a data processing system that includescomponents in addition to or in place of those illustrated for dataprocessing system 1100. Further, components shown in FIG. 11 may bevaried from the illustrative examples shown.

Illustrative embodiments of the disclosure may be described in thecontext of aircraft manufacturing and service method 1200 as shown inFIG. 12 and aircraft 1300 as shown in FIG. 13. Turning first to FIG. 12,an illustration of an aircraft manufacturing and service method in theform of a block diagram is depicted in accordance with an illustrativeembodiment. During pre-production, aircraft manufacturing and servicemethod 1200 may include specification and design 1202 of aircraft 1300in FIG. 13 and material procurement 1204.

During production, component and subassembly manufacturing 1206 andsystem integration 1208 of aircraft 1300 in FIG. 13 takes place.Thereafter, aircraft 1300 in FIG. 13 may go through certification anddelivery 1210 in order to be placed in service 1212. While in service1212 by a customer, aircraft 1300 in FIG. 13 is scheduled for routinemaintenance and service 1214, which may include modification,reconfiguration, refurbishment, and other maintenance or service.

Each of the processes of aircraft manufacturing and service method 1200may be performed or carried out by a system integrator, a third party,and/or an operator. In these examples, the operator may be a customer.For the purposes of this description, a system integrator may include,without limitation, any number of aircraft manufacturers andmajor-system subcontractors; a third party may include, withoutlimitation, any number of vendors, subcontractors, and suppliers; and anoperator may be an airline, a leasing company, a military entity, aservice organization, and so on.

With reference now to FIG. 13, an illustration of an aircraft in theform of a block diagram is depicted in which an illustrative embodimentmay be implemented. In this example, aircraft 1300 is produced byaircraft manufacturing and service method 1200 in FIG. 12 and mayinclude airframe 1302 with systems 1304 and interior 1306. Examples ofsystems 1304 include one or more of propulsion system 1308, electricalsystem 1310, hydraulic system 1312, and environmental system 1314. Anynumber of other systems may be included. Although an aerospace exampleis shown, different illustrative embodiments may be applied to otherindustries, such as the automotive industry.

Apparatuses and methods embodied herein may be employed during at leastone of the stages of aircraft manufacturing and service method 1200 inFIG. 12. In particular, virtual computer system 100 from FIG. 1 may beused to develop and test application software during any one of thestages of aircraft manufacturing and service method 1200. For example,without limitation, virtual computer system 100 from FIG. 1 may be usedto develop and test application software for installation on a hardwareplatform onboard aircraft 1300 during at least one of component andsubassembly manufacturing 1206, system integration 1208, routinemaintenance and service 1214, or some other stage of aircraftmanufacturing and service method 1200.

In one illustrative example, components or subassemblies produced incomponent and subassembly manufacturing 1206 in FIG. 12 may befabricated or manufactured in a manner similar to components orsubassemblies produced while aircraft 1300 is in service 1212 in FIG.12. As yet another example, one or more apparatus embodiments, methodembodiments, or a combination thereof may be utilized during productionstages, such as component and subassembly manufacturing 1206 and systemintegration 1208 in FIG. 12. One or more apparatus embodiments, methodembodiments, or a combination thereof may be utilized while aircraft1300 is in service 1212 and/or during maintenance and service 1214 inFIG. 12. The use of a number of the different illustrative embodimentsmay substantially expedite the assembly of and/or reduce the cost ofaircraft 1300.

Thus, the illustrative embodiments provide a method and apparatus forqualifying application testing tools using virtual machines, such asvirtual machine 112 in FIG. 1. For example, verifying applicationtesting tool 124 in FIG. 1 using tool qualifier module 122 that is runon virtual machine 112 in FIG. 1 avoids the need to requalifyapplication testing tool 124 at a later point in time when virtualmachine 112 is run on a different hardware configuration. One virtualmachine, such as virtual machine 112 in FIG. 1, which has been qualifiedto run on a physical computer system, may be used to run multipleapplication testing tools that have been previously qualified withrespect to the virtual machine without having to individually re-verifyeach application testing tool.

The description of the different illustrative embodiments has beenpresented for purposes of illustration and description, and is notintended to be exhaustive or limited to the embodiments in the formdisclosed. Many modifications and variations will be apparent to thoseof ordinary skill in the art. Further, different illustrativeembodiments may provide different features as compared to otherdesirable embodiments. The embodiment or embodiments selected are chosenand described in order to best explain the principles of theembodiments, the practical application, and to enable others of ordinaryskill in the art to understand the disclosure for various embodimentswith various modifications as are suited to the particular usecontemplated.

What is claimed is:
 1. An apparatus comprising: a physical computersystem comprising: a first processor of a first processor type; and afirst operating system of a first operating type running on the firstprocessor of the first processor type, a tool qualifier module that isrun on the physical computer system to verify an application testingtool using tool qualification data to qualify the application testingtool; a benchmark module that establishes a benchmark performance forthe physical computer system; and a virtual computer system having aconfiguration comprising: a second processor of a second processor type;a second operating system of a second operating type running on thesecond processor of the second operating type; a virtual machineemulating the first processor of the first processor type and running onthe second processor of the second processor type; and a third operatingsystem of the first operating type running on the virtual machine; aqualifier module for verifying the virtual machine to determine whetherthe performance of the virtual machine is consistent with the benchmarkperformance of the physical computer system within selected tolerancesto thereby form a qualified virtual machine such that transferring theapplication testing tool after the application testing tool has beenqualified from the physical computer system to the qualified virtualmachine on the virtual computer system does not require requalifying theapplication testing tool.
 2. The apparatus of claim 1 furthercomprising: the application testing tool, wherein the applicationtesting tool is run on the virtual machine using the third operatingsystem to determine whether the application program is certifiable. 3.The apparatus of claim 2, wherein the virtual computer system comprises:an interface connection to a hardware platform, wherein the applicationprogram is output over the interface connection for installation on thehardware platform in response to a determination that the applicationprogram is certifiable.
 4. The apparatus of claim 3, wherein the vehiclehardware platform is selected from one of an aircraft hardware platform,a spacecraft hardware platform, a watercraft hardware platform, or aground vehicle hardware platform.
 5. The apparatus of claim 2, whereinthe application testing tool uses application data that includes atleast one of application source code, regression test input data, andregression test output data to test the application program for use onthe vehicle hardware platform.
 6. The apparatus of claim 1, wherein thetool qualification data includes at least one of tool source data,regression test input data, and regression test output data.
 7. Theapparatus of claim 1, wherein the first processor type is different fromthe second processor type and the first operating type is different fromthe second operating type.
 8. A method for testing application softwarefor installation on a hardware platform using virtual computer systems,the method comprising: verifying an application testing tool using atool qualifier module running on a physical computer system having afirst processor of a first processor type and a first operating systemof a first operating type running on the first processor of the firstprocessor type to qualify the application testing tool; establishing abenchmark performance for the physical computer system; verifying avirtual machine in a virtual computer system using a qualifier module todetermine whether a performance of the virtual machine is consistentwith the benchmark performance of the physical computer system withinselected tolerances to thereby form a qualified virtual machine, whereinthe virtual computer system has a configuration that comprises a secondprocessor of a second processor type; a second operating system of asecond operating type running on the second processor of the secondprocessor type; the virtual machine emulating the first processor of thefirst processor type and running on the second processor of the secondprocessor type; and a third operating system of the first operating typerunning on the virtual machine; and transferring the application testingtool, after the application testing tool has been qualified, from thephysical computer system to the qualified virtual machine on the virtualcomputer system, wherein using the application testing tool on thevirtual computer system to test an application program that is to beinstalled on the hardware platform does not require requalifying theapplication testing tool.
 9. The method of claim 8 further comprising:running the application testing tool on the virtual machine emulatingthe first processor of the first processor type using the thirdoperating system of the first operating type to test whether theapplication program is certifiable.
 10. The method of claim 9 furthercomprising: connecting the virtual computer system to the hardwareplatform via an interface connection; and outputting the applicationprogram over the interface connection for installation on the hardwareplatform in response to a determination that the application program iscertifiable.
 11. The method of claim 10 further comprising: installingthe application program on the hardware platform, wherein the hardwareplatform is a vehicle hardware platform that is selected from one of anaircraft hardware platform, a spacecraft hardware platform, a watercrafthardware platform, or a ground vehicle hardware platform.
 12. The methodof claim 8, wherein verifying the application testing tool comprises:verifying the application testing tool using the tool qualifier moduleand tool qualification data includes at least one of tool source data,regression test input data, or regression test output data.
 13. Themethod of claim 8 further comprising: transferring the applicationtesting tool after the application testing tool has been qualified fromthe virtual machine on the virtual computer system to a qualified newvirtual machine that emulates the first processor of the first type on anew virtual computer system, wherein the new virtual computer system hasa third processor of a third processor type that is different from thesecond processor of the second processor type on the virtual computersystem.
 14. The method of claim 8 further comprising: creating thevirtual machine to emulate the first processor of the first processortype when the virtual machine is run on any processor of the firstprocessor type.
 15. The method of claim 8 further comprising: modifyingthe virtual machine to form a modified virtual machine that iscompatible with a third processor of a third processor type and to runon the third processor using another operating system of anotheroperating type.
 16. The method of claim 15 further comprising: verifyingthe modified virtual machine on the third processor of the thirdprocessor type.
 17. The method of claim 16 further comprising: runningthe application testing tool on the modified virtual machine, which isrunning on the third processor without needing to requalify theapplication testing tool.
 18. A method for testing application softwarefor installation on an aircraft hardware platform using virtual computersystems, the method comprising: verifying an application testing toolusing a tool qualifier module running on a physical computer systemhaving a first processor of a first processor type and a first operatingsystem of a first operating type running on the first processor of thefirst processor type to qualify the application testing tool;establishing a benchmark performance for the physical computer system;verifying a virtual machine in a virtual computer system using aqualifier module to determine whether a performance of the virtualmachine is consistent with the benchmark performance of the physicalcomputer system within selected tolerances to thereby form a qualifiedvirtual machine, wherein the virtual computer system has a configurationthat comprises a second processor of a second processor type; a secondoperating system of a second operating type running on the secondprocessor of the second processor type; the virtual machine emulatingthe first processor of the first processor type and running on thesecond processor of the second processor type; and a third operatingsystem of the first operating type running on the virtual machine; andtransferring the application testing tool, after the application testingtool has been qualified, from the physical computer system to thequalified virtual machine on the virtual computer system, wherein usingthe application testing tool on the second virtual computer system totest an application program that is to be installed on the aircrafthardware platform does not require requalifying the application testingtool. testing an application program for installation on the aircrafthardware platform using the application testing tool running on thevirtual computer system to determine whether the application program iscertifiable without requalifying the application testing tool for use onthe virtual computer system; and installing the application program onthe aircraft hardware platform in response to a determination that theapplication program is certifiable.
 19. The method of claim 18 furthercomprising: creating the virtual machine to emulate the first processorof the first processor type when the virtual machine is run on anyprocessor of the first processor type.